1. Field of the Invention
The present invention relates to an image processing device, an image processing method, and a computer program product.
2. Description of the Related Art
An augmented reality (AR) technology that adds a virtual object to the actual world so as to present information beyond reality is known. A method is known, as a popular method for implementing the AR technology, in which an AR marker that is printed on a sheet, or the like, is captured by a camera, the relative position and the relative orientation of the camera with respect to the AR marker is analyzed, and a virtual object is added to the captured image.
The AR technology that uses an AR marker can be easily implemented by using a device that has a low calculation power, such as a mobile terminal. Furthermore, the method for detecting the relative position and the relative orientation of the camera with respect to the AR marker is described in “‘development of the augmented reality system building tool ARToolKit’ the Institute of Electronics, Information and Communication Engineers, the technology society, pp 79 to 86, 2002-02”, or the like.
The process of the AR technology that uses the AR marker is roughly divided into an input process and an output process to a display. During the input process, the area of an AR marker is extracted from the captured image that is acquired from the camera so that the position in the three-dimensional space is estimated, and the size of the AR marker and data of the virtual object that is identified by using the AR marker are acquired by using the ID that is obtained by analyzing the AR marker. During the output process to the display, the virtual object's data, which is acquired during the input process, is used to locate it in the three-dimensional space, and the data that is located in the three-dimensional space is drawn by a three-dimensional graphics processor, or the like, so that it is displayed on the display.
The above-described AR technology allows a user to have a simulated experience by making the thing that is difficult to physically create, such as a small vehicle or a dinosaur, look as if it exists in the place or by virtually locating a piece of furniture at a desired place in the home before buying it. Furthermore, the above-described AR technology allows a user to have various experiences by displaying different virtual objects depending on the time, user, or the like, without remaking the physical background.
A device that displays the result of drawing using the AR technology is, for example, a mobile phone. Furthermore, it may be, for example, a PC display, or a small terminal that is called a tablet and that includes a liquid crystal panel display. Furthermore, it may be, for example, a device that is called a head-mounted display and that includes a display that is secured to the head, or the like, and is in the vicinity of the eye. Here, the vicinity also includes the case where a mirror or a half mirror is installed in the path of light that enters the eye from the display. Contrary to a mobile phone, or the like, if a head-mounted display is used, a user does not need to hold the terminal by the hand; therefore, there is the advantage of free hands.
A printed AR marker is used for detecting the position where a virtual object is drawn. Therefore, if the sheet on which an AR marker is printed is moved or rotated, the drawing position on the display is changed in synchronization with the movement. Thus, the AR technology that uses the AR marker makes it possible to partially realize an interaction on a virtual object in accordance with a human operation. However, it is difficult to realize detailed interactions, such as flipping through the pages of a book that is displayed as a virtual object, or opening a box that is displayed as a virtual object. That is, conventional AR technologies that use AR markers make it possible to simply move a virtual object itself.
Therefore, if the AR technology allows a user to perform more detailed operations on the virtual object that is displayed on the display in a superimposed manner and to realize an interaction, it is possible to provide a user with more varieties of virtual experiences.
For example, according to the technique that is described in Japanese Patent Application Laid-open No. 2012-212343, if a virtual object is displayed in the captured image in a superimposed manner on the basis of a physical object that is captured by the camera, a still image is clipped so that the virtual object is continuously displayed. Then, the display of the virtual object is changed in accordance with a user's touch operation, key operation, or the like. However, according to this technique, a device, such as a touch panel, is necessary in addition to a camera to detect a user's operation.
Furthermore, the device that recognizes the movement, called a gesture, of a human hand, or the like, from the moving image that is captured by a camera so as to recognize a user's operation is conceivable as an implementation means. In this case, a device, such as a touch panel, is not necessary; however, it is difficult for a user to know what kind of operation the user can perform or what kind of gestures the system can recognize. Furthermore, the accuracy with which gestures are recognized is relatively low.
A relevant technique is also described in Japanese Patent No. 4977725.
In view of the foregoing, there is a need to provide an image processing device, an image processing method, and a computer program product that easily realize an interaction in accordance with a user's operation when the AR technology is implemented by using images that are captured by a camera.